Vision-controlled system and method thereof

ABSTRACT

A vision-controlled system and method thereof for detecting changes in a monitoring environment are disclosed. The system includes an image-capturing device for capturing an image; an image analyzer for receiving the captured image and analyzing the same to provide a control signal; and a controller coupled to the image analyzer for receiving the control signal and processing the same.

FIELD OF INVENTION

The present invention relates generally to controller systems. Morespecifically, the present invention relates to control systems based oninput binary images.

BACKGROUND OF INVENTION

It is desirable to have control systems that are cost effective and canbe deployed for a number of applications. However, most existing controlsystems are dedicated systems comprising a plurality of dedicated andexpensive components. One such control system is a vision system. Thevision system used for monitoring an object or environment providesinformation without physical contacts with the objects or environmentbeing monitored. The vision system typically concerns with imageprocessing which tends to involve complex calculation. Disadvantageswith complex calculation include making use of large computer memory andhigh-speed computing processor in order to provide real-time responses.

Therefore, it is desirable to provide a vision-controlled system thatrequires substantially less computer memory and nominal speed computingprocessor.

SUMMARY OF INVENTION

A vision-controlled system and method thereof for detecting changes in amonitoring environment are disclosed. In accordance with a first aspectof the invention, the system comprises an image-capturing device forcapturing an image; an image analyzer for receiving the captured imageand analyzing the same to provide a control signal; and a controllercoupled to the image analyzer for receiving the control signal forcontrolling actuators. In accordance with a second aspect of theinvention, the method comprises receiving an image; comparing the imagewith a reference image; and generating a control signal based on theoutcome of the comparison for controlling actuators.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention are herein described, purely by way ofexample, with references made to the accompanying drawings, in which:

FIG. 1 shows a vision-controlled system according to an embodiment ofthe present invention;

FIG. 2 shows an example of a captured image according to an embodimentof the present invention;

FIG. 3 shows an example of a user interface of the vision-controlledsystem according to an embodiment of the present invention; and

FIG. 4 shows a flowchart depicting steps for working thevision-controlled system according to an embodiment of the presentinvention.

DETAILED DESCRIPTION OF INVENTION

A vision-controlled system 100 capable of simultaneously monitoringmultiple objects and controlling multiple actuators in response to anychanges to or movements of the observed objects is shown in FIG. 1. Thevision-controlled system 100 includes an image-capturing device 110, animage analyzer 130, a controller 140 and actuators 150. Examples of theactuators 150 include one or more robot 152 and one or more finalcontrol elements 154. The final control elements 154 are ON-OFF devicesand are responsive to signals from the controller 140. Theimage-capturing device 110 can be any of known photo sensors such as adigital camera. In an embodiment, the image-capturing device 110 is alow cost, low resolution camera, the like of a web-cam, which has aresolution of 352×288×100,000 pixels and capable of capturing images at25 frames or more per second. The rate of frame capture determines thefrequency or interval at which any changes to the observed objects isanalyzed. Thus, the rate of frame capture has bearing on theresponsiveness of the vision-controlled system 100 toward any changes tothe observed objects in the image frame.

The image-capturing device 110 also includes a digitizer 120 forconverting the captured images into equivalent binary images. That is,each pixel of the captured image is rendered black (e.g. a “1”) or white(e.g. a “0”) depending on the color depth of each pixel of the(original) captured image. Processing binary image provides for simplepixel manipulation and rapid identification as each pixel is eitherblack or white, thus, resulting in substantially faster processing andrequiring smaller memory compared to processing a gray scaled or coloredimage. Further, the binary image alleviates the need for a high-speedcomputer-processing unit.

The binary image is fed to the image analyzer 130 where portions of thebinary image are analyzed in accordance with a reference image, which isalso in the binary format. The binary image can be temporally stored ina memory 132, where the reference image is also stored. The referenceimage is an image of the observed objects in an initial or desiredstate. As such, the reference image is divided into different portionswith each portion defining a position (or location) of an object ofinterest in the image to be observed. FIG. 2 shows an example of acaptured frame 200 having three observable objects 210, 215 and 220. Thepixels making up the frame 200 are arranged in m columns by n rows.Thus, each pixel in the frame 200 can be identified by a coordinate (m,n). In an embodiment, two objects of interest are objects 210 and 220.Object 210 can be defined by portion 212, which is made of sixneighboring pixels. Similarly, object 220 can be defined by portion 222,which is made of two neighboring pixels. In an alternative embodiment,object 210 can be defined by portion 214, which is made of one pixel;and object 220 can be defined by portion 224, which is made of onepixel.

Each portion defining the object is given a reference value. In anembodiment, the reference value can be the average of the binary valuesof all pixels within the portion. For example, in portion 212, pixels(9,1) and (10,1) are white thus having values zeros and pixels (9,2),(10,2), (9,3) and (10,3) are black thus having values ones. Therefore,the average reference value of portion 212 is two-third (i.e.(1+1+1+1+0+0) divided by 6 pixels). In the case where portion 212 ismade of one pixel 214 (at coordinate (9,2)), the reference value is onesince pixel 214 is black. It should be noted that this is just oneexemplary method of determining the values of the portions.

The image analyzer 130 compares values of the portions of the binaryimage with corresponding reference values of the corresponding portionsof the reference image. A change in the value of any of the portions ofthe binary image indicates that the object defined by the portion iseither changed or displaced. In an embodiment, if there is no change inthe value of any of the portions of the binary image, no signal isoutput to the controller 140. In an alternative embodiment, for eachanalyzed portion of the binary image, if there is a change in the value,an active signal is output to the controller 140. However, if there isno change in the value, a non-active signal is output to the controller140. The active signal indicates to the controller 140 that the state ofthe observed object has changed from its initial state. Conversely, thenon-active signal indicates to the controller 140 that the state of theobserved object remains unchanged. In an embodiment, the active andnon-active signals also include information indicating which object thesignals are concerned with. In another embodiment, the active andnon-active signals can include information indicating which of theactuators 150 are the signals intended for.

In another embodiment, the image analyzer 130 can be instructed toobserve portions (or regions) of the binary image where there is noobject. In this case, the region of interest in the binary image ismerely made up of background images. Thus, when a region of interest isobstructed by a foreground object, for example, an object passing by oran object is placed in the observed region, a change detection is made.This change in the observed region can be used to trigger the actuators150.

In an embodiment, signals received at the controller 140 are amplified(142) and fed to a switching circuit 144 where the actuators aretriggered. The actuators 150 can be coupled to the controller 140 viawired communication connections 146 or wireless communicationconnections 148 or a combination thereof. Examples of wiredcommunication connections 146 include LAN connections or directionconnection such as parallel cable, serial cable, USB connection andFireWire® connection. Examples of wireless communication connections 148include wireless LAN, Bluetooth®, Wi-Fi, and the like wirelesscommunication protocols.

The selection and usage of the type of actuators 150 is dependent on thenature of the applications. Examples of possible applications includejewel security, where many items can be monitored and multiple alarmscan be triggered simultaneously, security systems in museums, automaticsimultaneous fluid level control, and robotics control in manufacturingenvironments. Thus, in an embodiment, the number of objects observablerelates closely to the number of pixels in each captured image frame.

According to an embodiment, the image-capturing device 110 is equippedwith a motion mechanism (not shown), which is responsive to feedbacksignals 115 from the controller 140. The feedback signals 115 are forcontrolling the movement of the image-capturing device 110 to change theviewing angle thereof. The feedback signals 115 can be transmitted fromthe controller 140 to the image-capturing device 110 either by using awired communication connection or a wireless communication connection.

In an embodiment, the vision-controlled system 100 also includes a userinterface 160 coupled to the image analyzer 130. Images output from thedigitizer 120 can be displayed on the user interface 160 to provide avisual feedback to operators. Among other things, the user interface 160also enables the operators to identify objects of interest in the images(i.e. object selection) and linking each of the identified objects tocorresponding actuators 150. An exemplary user interface 300 is shown inFIG. 3. The user interface 300 includes a start button 305, an exitbutton 307, an image window 310 where the images output from thedigitizer 130 are displayed and a data-logging window 320 where anychanges to the values of the portions of the binary image are logged.Upon pressing the start button 305, the image-capturing device 110captures an image, which is processed by the digitizer 120 andsubsequently displayed in the image window 310. The user interface 300also includes controls for identifying new objects (ADD POS button 330)in the binary image to be monitored and removing objects (DEL POS button340) from being monitored. Upon pressing the ADD POS button 330, theoperator is provided with an option to identify an object of interest inthe displayed image for monitoring. The option can be provided in theform of a changed cursor to indicate object selection mode. The objectis identified by the operator selecting a portion of the displayed imagethat represents the object of interest. As described in the foregoing,the portion can be a group of neighboring pixels or a single pixel. Thenewly selected portion is given a name and thereafter listed in thedata-logging window 320 along with the value (reference value) of theselected portion. The operator is also provided with an option to linkthe newly selected portion to one or more actuators. In an embodiment,the user interface 300 also includes a motion controller 350 formanually controlling the robots 152 or the viewing angle of theimage-capturing device 110.

Further, a specific application control 360 can be provided to allow forspecialized applications such as object trajectory tracking, automaticpath-detection driving and virtual tele-surgery simulation. The specificapplication control 360 includes application selection options 362, anapplication start button 364 and an application stop button 366. In theexemplary example of the object trajectory tracking, once the operatorhas selected the trajectory tracking application, the operation isrequested to define the object intended for tracking. In this case, theobject is defined by selecting two portions of the pixels of the image(displayed in the image window 310) corresponding to two extreme ends ofthe object. It should be noted that more than two portions of the objectoutline can be defined and that each defined portion can consists ofeither one pixel or a number of neighboring pixels. Upon pressing theapplication start button 364, the image-capturing device 110 tracks themovements of the object by analyzing the defined portions of the pixelsand sending corresponding signals to the controller 140 to control themovement of the image-capturing device 110 via feedback signals 115.Tracking the movements of the object can be achieved by using an ‘AND’logic to ensure that the define portions are consistently dark, whichindicates the presence of the object. For example, when the value of theright extreme end of the object changes state (i.e. becomes white), itindicates that the object has moved to the right. Thus, a correspondingfeedback signal is to control the image-capturing device 110 to move anincrement to the right to compensate for the movement of the object. Insimilarly manner, the movements of the object to the left, up and downcan also be detected and compensated. Pressing the application stopbutton 366 halts the object movement tracking.

FIG. 4 shows a flowchart 400, according to an embodiment, depictingsteps of working the vision-controlled system 100. In step 402, an imageof interest is provided for use as a reference image. The referenceimage is digitized in step 404. The digitizing step 404 involvesconverting the input reference image into a reference binary imagewherein each pixel therein is rendered either black or white dependingon the color depth of the pixel. The reference binary image issubsequently displayed on a user interface (such as the user interface300 shown in FIG. 3) so that an operator can identify objects capturedin the reference binary image for monitoring purposes. In step 406, theoperator identifies an object of interest by selecting a portion of thereference binary image representing the object. The portion, which is areference portion representing the object of interest, can be either apixel or multiple pixels in size. Once the reference portion isidentified, a reference value based on the reference portion isdetermined in step 408. In an embodiment, as each pixel is with black orwhite, a simple way for determining the value of the reference portionis to count the number of black and white pixels in the portion. Inanother embodiment, the value of the reference portion can be an averageof the binary values of all the pixels in the reference portion. In step408, the identified reference portion can be linked to one or multipleactuators. The reference value, reference portion and identities ofactuators linked thereto are then stored in a memory along with thereference portion in step 410.

If there are more than one objects of interest, more reference portionscan be selected from step 406. Once the objects of interest areidentified, the monitoring process can commence. Typically, thisinvolves providing a fresh image in step 420. The image is subsequentlydigitized in step 422 to provide a binary image. The binary image issubsequently analyzed in step 424. In this step 424, portions of thebinary image are identified in accordance with the reference portionsstored in the memory. Values of the identified portions are subsequentlydetermined and compared with the corresponding reference values storedin the memory. Depending on the outcome of the comparison, correspondingcontrol signals are generated in step 426. In an embodiment, if a valueof an identified portion is substantially the same as a correspondingreference value, no control signal need to be generated. However, if thevalue of the identified portion is substantially different from thecorresponding reference value, an active signal is generated. The activesignal indicates a change in the observed area or object has occurred.In an alternative embodiment, if the value of the identified portion issubstantially the same as the corresponding reference value, anon-active signal is generated; otherwise, an active signal isgenerated. The generated signals are subsequently sent to correspondingactuators to trigger the actuators to perform a pre-determined task instep 428. In step 426, once the control signals are generated, themonitoring process can be repeated from step 420 until it is terminated.

While the present invention has been described in detail with respect tospecific embodiments thereof, it will be appreciated that those skilledin the art, upon attaining an understanding of the foregoing may readilyadapt the present technology for alterations to, variations of, andequivalents to such embodiments. Accordingly, the scope of the presentdisclosure is by way of example and is not to be limited to the detailsgiven herein, but may be modified within the scope and equivalents ofthe appended claims.

1. A system comprising: an image-capturing device for capturing an imageand converting the captured image into a binary image; an image analyzerfor receiving the binary image and analyzing the binary image bycomparing the binary image with a reference binary image stored in amemory to generate a control signal, wherein comparing the binary imagewith the reference binary image comprises: selecting a portion of thebinary image based on a reference portion of the reference binary image,wherein the portion comprises a plurality of pixels, each pixel having apixel value; determining a portion value of the selected portion,wherein the portion value being an average of the pixel value of theplurality of pixels within the portion; and comparing the portion valuewith a reference portion value of the reference portion; and acontroller coupled to the image analyzer for receiving the controlsignal and processing the same, wherein the image-capturing devicecomprises a motion mechanism responsive to a feedback signal provided bythe controller for controlling movement of the image-capturing device.2. The system of claim 1, wherein the image-capturing device is aweb-cam.
 3. The system of claim 1, wherein the image-capturing devicecomprises a digitizer for providing the binary image based on thecaptured image.
 4. A method for controlling movement of animage-capturing device and one or more actuators, comprising: capturingan image by the image-capturing device and converting the captured imageinto a binary image; receiving the binary image and analyzing the binaryimage by comparing the binary image with a reference binary image storedin a memory to generate a control signal by an image analyzer, whereincomparing the binary image with the reference binary image comprises:selecting a portion of the binary image based on a reference portion ofthe reference binary image, wherein the portion comprises a plurality ofpixels, each pixel having a pixel value; determining a portion value ofthe selected portion, wherein the portion value being an average of thepixel value of the plurality of pixels within the portion; and comparingthe portion value with a reference portion value of the referenceportion; and receiving the control signal and processing the controlsignal by a controller, wherein the image-capturing device comprises amotion mechanism responsive to a feedback signal provided by thecontroller for controlling the movement of the image-capturing device.5. The method of claim 4, wherein the pixel value being a ONE or ZERO.6. The method of claim 4, wherein generating the control signalcomprises: providing an active signal upon finding the portion valuesubstantially different from the reference portion value; and providinga non-active signal upon finding the portion value substantially same asthe reference portion value.
 7. The system of claim 1, wherein the imageanalyzer generates the control signal based on the outcome of thecomparison.
 8. The system of claim 1 further comprising a plurality ofactuators coupled to the controller, the controller comprising aswitching circuit for simultaneously controlling the plurality ofactuators based on the control signal.
 9. The system of claim 1 furthercomprising an actuator coupled to the controller, wherein the controllersends the control signal to the actuator to trigger the actuator toperform a pre-determined task.
 10. The system of claim 9, wherein theactuator comprises a robot.
 11. The system of claim 9, wherein theactuator comprises a final control element.
 12. The system of claim 9,wherein the actuator being coupled to the controller using a wiredconnection.
 13. The system of claim 9, wherein the actuator beingcoupled to the controller using a wireless connection.
 14. The system ofclaim 1 further comprising a user interface having an image window fordisplaying the image.
 15. The system of claim 14, wherein the userinterface further comprising controls for establishing referenceportions of the image for use by the image analyzer.
 16. The system ofclaim 1, wherein generating the control signal comprises: providing anactive signal upon finding the portion value substantially differentfrom the reference portion value; and providing a non-active signal uponfinding the portion value substantially same as the reference portionvalue.
 17. The system of claim 1, wherein the feedback signal istransmitted from the controller to the image-capturing device using awired communication connection or a wireless communication connection.18. The method of claim 4, wherein the control signal is generated basedon the outcome of the comparison.
 19. The method of claim 4, wherein thefeedback signal is transmitted from the controller to theimage-capturing device using a wired communication connection or awireless communication connection.
 20. The method of claim 4 furthercomprising activating a plurality of actuators simultaneously based onthe control signal.
 21. The method of claim 4 further comprisingactivating an actuator using the control signal.
 22. The method of claim21, wherein activating the actuator comprises sending the control signalto the actuator using a wired connection or a wireless connection totrigger the actuator to perform a pre-determined task.
 23. The method ofclaim 4 further comprising providing a user interface, the userinterface comprising an image window for displaying the image, andcontrols for establishing reference portions of the image for use by theimage analyzer.